259 research outputs found
Pseudo-Marginal Bayesian Inference for Gaussian Processes
The main challenges that arise when adopting Gaussian Process priors in
probabilistic modeling are how to carry out exact Bayesian inference and how to
account for uncertainty on model parameters when making model-based predictions
on out-of-sample data. Using probit regression as an illustrative working
example, this paper presents a general and effective methodology based on the
pseudo-marginal approach to Markov chain Monte Carlo that efficiently addresses
both of these issues. The results presented in this paper show improvements
over existing sampling methods to simulate from the posterior distribution over
the parameters defining the covariance function of the Gaussian Process prior.
This is particularly important as it offers a powerful tool to carry out full
Bayesian inference of Gaussian Process based hierarchic statistical models in
general. The results also demonstrate that Monte Carlo based integration of all
model parameters is actually feasible in this class of models providing a
superior quantification of uncertainty in predictions. Extensive comparisons
with respect to state-of-the-art probabilistic classifiers confirm this
assertion.Comment: 14 pages double colum
Information-geometric Markov Chain Monte Carlo methods using Diffusions
Recent work incorporating geometric ideas in Markov chain Monte Carlo is
reviewed in order to highlight these advances and their possible application in
a range of domains beyond Statistics. A full exposition of Markov chains and
their use in Monte Carlo simulation for Statistical inference and molecular
dynamics is provided, with particular emphasis on methods based on Langevin
diffusions. After this geometric concepts in Markov chain Monte Carlo are
introduced. A full derivation of the Langevin diffusion on a Riemannian
manifold is given, together with a discussion of appropriate Riemannian metric
choice for different problems. A survey of applications is provided, and some
open questions are discussed.Comment: 22 pages, 2 figure
MCMC inference for Markov Jump Processes via the Linear Noise Approximation
Bayesian analysis for Markov jump processes is a non-trivial and challenging
problem. Although exact inference is theoretically possible, it is
computationally demanding thus its applicability is limited to a small class of
problems. In this paper we describe the application of Riemann manifold MCMC
methods using an approximation to the likelihood of the Markov jump process
which is valid when the system modelled is near its thermodynamic limit. The
proposed approach is both statistically and computationally efficient while the
convergence rate and mixing of the chains allows for fast MCMC inference. The
methodology is evaluated using numerical simulations on two problems from
chemical kinetics and one from systems biology
Bayesian Quadrature for Multiple Related Integrals
Bayesian probabilistic numerical methods are a set of tools providing
posterior distributions on the output of numerical methods. The use of these
methods is usually motivated by the fact that they can represent our
uncertainty due to incomplete/finite information about the continuous
mathematical problem being approximated. In this paper, we demonstrate that
this paradigm can provide additional advantages, such as the possibility of
transferring information between several numerical methods. This allows users
to represent uncertainty in a more faithful manner and, as a by-product,
provide increased numerical efficiency. We propose the first such numerical
method by extending the well-known Bayesian quadrature algorithm to the case
where we are interested in computing the integral of several related functions.
We then prove convergence rates for the method in the well-specified and
misspecified cases, and demonstrate its efficiency in the context of
multi-fidelity models for complex engineering systems and a problem of global
illumination in computer graphics.Comment: Proceedings of the 35th International Conference on Machine Learning
(ICML), PMLR 80:5369-5378, 201
- …